This application is based upon application No. 11-100635 filed in Japan, the contents of which are hereby incorporated by reference.
1. Field of the Invention
The present invention relates to a camera for getting information upon three-dimensional shape of an object.
2. Description of the Related Arts
Conventionally, as a device for getting information upon three-dimensional shape of an object, for example, there has been proposed a device for getting the information from a pair of images that have passed through a plurality of photographing lenses, and there has conventionally been proposed a device for detecting distance distribution by projecting light onto an object (or subject) on the basis of the principle of triangulation, as shown in FIG. 23.
Also, for example, there has been proposed a device for detecting distance distribution by projecting a fringe pattern onto an object and by inputting the fringe pattern by another camera, on the basis of the principle of triangulation, as disclosed in Japanese Laid-Open Patent Publication No. 6-249624.
Also, for example, there has conventionally been proposed a device to seek for an undulation (or up and down) of an object by projecting a lattice pattern onto the object and by observing the object in different angles, in which modified data corresponding to the modified lattice pattern formed in compliance with the undulation of the object is gained (Journal of the Institute of Precision Engineering, 55, 10, 85 (1989)).
Also, for example, there has conventionally been proposed a device for measuring optical distribution by projecting a gray-code pattern with a CCD camera, instead of projecting the lattice pattern, as shown in FIG. 24.
In order to gain the information upon the three-dimensional shape with the aforementioned device, however, there is need of taking (or photographing) a plurality of images, and the process, subsequent to taking the images, of the information is complex. As a result, a longer time is required for an operation to take an image of the object, and/or a longer time is required for processing the information after taking the image. In other words, it is considered that there is no special problem if the mechanism of the aforementioned device is employed for any measuring device; however, it is considered that it is not suitable to apply the mechanism thereof to any camera.
Therefore, as a device for speedily getting the information upon three-dimensional shape with higher precision, for example, there has been proposed the following device therefor.
Namely, as shown in FIG. 25 (Source: xe2x80x9cOptical Three-Dimensional Measurement,xe2x80x9d edited by Toru Yoshizawa, Shin-Gijutsu Communications (or New Technical Communications), page 89, FIGS. 5.2.12a), a stripe pattern is firstly projected onto an object (or subject); and the stripe pattern formed on the object is detected at an angle determined in terms of a design relative to the projected stripe pattern; and then the distance distribution of the object is detected from the deformed image of the stripe pattern in compliance with the unevenness (or irregularity) of the object. That is, with respect to a phase of an image measured at each image point, a shift (or swerve, or change) in phase relative to its original stripe is computed. The phase shift includes information upon the height of the object. In this relation, the distance distribution of the object is sought, or determined, from the phase information and its triangulation information. However, there is need of high precision for the detection. Because there is a limitation in density distribution of the stripe pattern or in light intensity thereof, there has been employed a method for seeking the distance distribution of the object by a plurality of images taken with the position of the stripe pattern being shifted bit by bit. Based upon this method, for example, stripe patterns with four shifts in phase of 0xc2x0, 90xc2x0, 180xc2x0 and 270xc2x0, are projected.
By the way, as a measuring device, there has been conventionally provided a commercial product in which a slitting light is scanned. According to the measuring device, it takes several hundreds of milliseconds to scan the slitting light for the purpose of getting the information upon three-dimensional shape information.
Meanwhile, there has conventionally been provided a digital camera in which a plurality of images are consecutively taken in case that the photographing mode is a successively photographing mode and in which the image is recorded (or stored) upon a memory card after the plurality of images are taken (or after the object is successively photographed). However, the digital camera is not a camera for getting information upon three-dimensional shape of an object.
According to the aforementioned device and method, however, if the stripe pattern which has a plurality of stripes has only one cycle, the density distribution becomes so rough that it is not possible to heighten the precision for getting the information upon three-dimensional shape of an object.
In order to solve this technical problem, there have conventionally been taken measures in which a stripe pattern with several cycles is projected onto the object. In this case, however, if the object has a deep distance distribution, it is not possible to identify which particular stripe of the stripe pattern a particular piece of information upon the stripe of the stripe pattern corresponds to. Namely, in this case, the angle used in the triangulation may be interpreted as a different angle (or a wrong angle), by taking a particular stripe of the stripe pattern for a strip thereof corresponding to a different ordinal number (or a wrong ordinal number) of the plurality of stripes thereof. As a result, a wrong distance distribution is gained.
More specifically, for example, as shown in FIG. 28, a point xe2x80x9cAxe2x80x9d which locates on a fore plane xe2x80x9cP1xe2x80x9d becomes a piece of information upon the third stripe of the plurality of stripes of the stripe pattern; on the other hand, a point xe2x80x9cBxe2x80x9d which locates on a rear plane xe2x80x9cP2xe2x80x9d becomes another piece of information upon the fourth stripe thereof. However, if it is not possible to accurately identify which ordinal number of stripe the particular stripe of the plurality of stripes of the stripe pattern corresponds to, the points xe2x80x9cAxe2x80x9d and xe2x80x9cBxe2x80x9d can not be distinguished from each other, only on the basis of information upon the image sensed by its light receiving part.
Furthermore, in case of increasing the number of points for detection by increasing the number of stripes in a particular stripe pattern for the purpose of enhancing the precision in distance distribution, if a particular stripe is wrongly identified in its ordinal number of stripes, it also leads to a false detection of the information upon three-dimensional shape of the object.
Under such a circumstance, for the purpose of reducing the misdetection of the information thereupon, and for the purpose of increasing the accuracy for identifying any particular stripe of the plurality of stripes of the stripe pattern, applicant has proposed the following improvement of the stripe pattern to be projected (see, for example, Japanese Patent Application No. 11-87124, not laid open yet).
Namely, according to the improvement, for example, a pattern having a plurality of frequencies, and/or a pattern having a plurality of codes and stripes, and/or a light having a colored pattern, is/are projected onto an object. Further, according to the improvement, not only such a pattern with a gradation, but also a marker to be able to identify a particular position or location on the pattern, are projected onto the object.
According to the aforementioned device and the improvement, however, if it is not possible to accurately identify which ordinal number of stripe of the plurality of stripes forming the stripe pattern a particular stripe corresponds to, wrong data, or a piece of wrong information upon three-dimensional shape of the object, is gained. Namely, no matter which types of patterned lights may be projected onto the object, there exists a possibility that it is difficult to accurately identify which ordinal number of stripe thereof a particular stripe to be identified corresponds to, depending upon a situation, or a condition, of the object onto which the patterned light is projected.
More specifically, for example, depending upon a color of the object and/or upon a color of a patterned light which is projected onto the object, there may exist a possibility that it is not possible to identify a position of a particular stripe of the plurality of stripes, in an attempt of identifying the position thereof by making use of a color or colors. Also, for example, it may be difficult to identify a position of a particular stripe to be identified of the plurality of stripes, if the space frequency of a pattern (or design or appearance) of an object is higher, or if the space frequency thereof of the object is similar, or the same, to the frequency of the stripe pattern of the patterned light projected onto the object.
Consequently, it is desired to be able to accurately get information upon three-dimensional shape of an object, even if the color of the object and/or the color of the light with which the object is lit, are/is biased, or offset, to a particular color, and/or even if the space frequency of the object is similar, or the same, to the frequency of the stripe pattern of the patterned light projected onto the object.
Therefore, it is an object of the present invention to provide a camera which is able to accurately get information upon three-dimensional shape of an object, without its being affected by the color of the object and/or by the color of the light with which the object is lit, and/or by the space frequency of the object.
In accomplishing this and other objects of the present invention, according to an aspect thereof, there is provided a camera for getting information upon three-dimensional shape, comprising: an image taking device for taking an image of an object within a region to be photographed; a projector which is capable of selectively projecting one of a plurality of types of pattern lights on the region, in which the image taking device takes a projection pattern which is formed on the object within the region by the one thereof; and a detector for detecting a photographing condition of one of the object, a light with which the object is lit, and both of the object and the light, wherein the projector selectively projects the one of the pattern lights, on a basis of a result of detection, by the detector, of the photographing condition thereof.
In the construction, the projector selects the one to project of the pattern lights, on the basis of the result of detection, by the detector, of the photographing condition thereof, so as to be able to accurately gain the information upon the three-dimensional shape of the object. That is, in case that there is a possibility that the information thereupon can not be gained from the projection pattern which is formed on the object within the region, namely, in case that the color of the pattern light projected from the projector is similar, or the same, to the color of the object itself and/or is similar, or the same, to the color of the light with which the object is lit, and/or in case that the frequency of the pattern light is similar, or the same, to the frequency (or space frequency) of the pattern (or design or appearance or look) of the object itself, then the pattern light is switched over to another pattern which allows the accurate information upon the three-dimensional shape of the object to be gained.
Therefore, according the construction, it is possible to precisely gain the information upon the three-dimensional shape of the object, without its being affected by the color of the object and/or by the color of the light with which the object is lit, and/or by the object itself (or by the space frequency of the object).
According to an embodiment, the detector detects a color of the object, as the photographing condition.
That is, for example, in case that the color of the pattern light projected from the projector is similar, or the same, to the color of the object, there is a possibility that the projection pattern, which is formed on the object, can not be detected from the image of the object, or there is a possibility that the projection pattern is detected erroneously, or inaccurately, from the image of the object. According to the aforementioned construction, however, on the basis of the result of detection, by the detector, of the color of the object, the projector selectively projects a particular type of a pattern light, selected from the pattern lights, which allows the projection pattern formed on the object to be accurately detected.
According to an embodiment, the detector detects a spatial frequency of the object, as the photographing condition.
That is, for example, in case that a dark part of the projection pattern formed on the object overlaps a dark part of the object itself, there is a possibility that the projection pattern, which is formed on the object, can not be detected from the image of the object, or there is a possibility that the projection pattern is detected erroneously, or inaccurately, from the image of the object. According to the aforementioned embodiment, however, on the basis of the result of detection, by the detector, of the space frequency of the object, the projector selectively projects a particular type of a pattern light, selected from the pattern lights, which allows the projection pattern formed on the object to be accurately detected.
According to an embodiment, the detector detects a color of the light, as the photographing condition.
That is, for example, in case that a color of the light (for example, available light or stationary light or natural light) with which the object is lit is similar, or the same, to a color of the pattern light projected from the projector, there is a possibility that the projection pattern with the same color as that of the light, which is formed on the object, can not be detected from the image of the object, or there is a possibility that the projection pattern is detected erroneously, or inaccurately, from the image of the object. According to the construction of this embodiment, however, on the basis of the result of detection, by the detector, of the color of the light, the projector selectively projects a particular type of a pattern light, selected from the pattern lights, which allows the projection pattern formed on the object to be accurately detected.
According to an embodiment, there is further provided a sensor for auto-white balance, wherein the detector detects the photographing condition, on a basis of an output from the sensor for auto-white balance.
That is, generally, the sensor for auto-white balance is employed for compensating a bias, or an offset, of a color of an image being photographed, for example, in a video camera, and this sensor can also be employed in the aforementioned construction. For example, the sensor for auto-white balance can be provided, being separate from the image taking device. As a modification, the sensor for auto-white balance can be employed as an image taking element of the image taking device. The detector detects the bias, or offset, of a color of an image being photographed, on the basis of the output from this sensor for auto-white balance.
That is, for example, in case that a color included in the pattern light projected from the projector is similar, or the same, to a main component color which causing the bias, or the offset, of the color of the image being photographed, there is a possibility that the projection pattern formed on the object can not be detected from the photographed image with the pattern of the object, or there is a possibility that the projection pattern is detected erroneously, or inaccurately, from the photographed image with the pattern thereof. According to the construction of this embodiment, however, on the basis of the result of detection, by the detector, of the bias, or offset, of the color of the image being photographed, the projector selectively projects a particular type of a pattern light, selected from the pattern lights, which allows the projection pattern formed on the object to be accurately detected.
According to an embodiment, the plurality of types of pattern lights are different to each other at least in color distribution.
According to the construction, it is possible to switch over from one pattern light with a color to another pattern light with another color. With the construction, for example, in compliance with the color of the object and/or the color of the light with which the object is lit, it is possible to project a suitable pattern light with a particular color thus selected, so as to allow the precise information upon the three-dimensional shape of the object to be gained.
According to an embodiment, the plurality of types of pattern lights are different to each other at least in frequency of illuminance distribution.
According to the construction, it is possible to switch over from one pattern light with a frequency to another pattern light with another frequency. With the construction, for example, in compliance with the frequency included in the pattern (or design or appearance) of the object, it is possible to project a suitable pattern light with a particular frequency thus selected, so as to allow the precise information upon the three-dimensional shape of the object to be gained.
In accomplishing the aforementioned objects of the present invention, according to another aspect thereof, there is provided a camera, comprising: a projector which is capable of selectively projecting a light having a first pattern and a light having a second pattern being different from the first pattern, on an object; a selector for selecting one of the first pattern and the second pattern; a photographing part for taking an image of the object which is projected with the light, in which the photographing part is separated from the projector with a predetermined base length between the photographing part and the projector; and an outputting part for outputting data of three-dimensional shape of the object, on a basis of a result of photographing.